Eco-Friendly Fast Lane: Go, Rust & Swift Are Leaving PHP, Node.js And Python In The Dust

When I had to write a performance critical part of a web app around 15 years ago, my only serious option was C/C++. It was an application that had to process tens of thousands of files in a relatively short period and store them on multiple Linux servers. Back then, the decision to write a web application in C/C++ was a well-considered decision. I would only write web apps in C/C++ when there was no way around it. The complexities of multi threading and memory management in C/C++ meant development took way longer than in any other programming language.

Compiled vs. interpreted languages: an uneven battle in the Internet skies?

Today, with the arrival of Go, Rust and Swift, writing highly efficient high performance code doesn’t take that long anymore. In fact, I write even the simplest apps on the server in Go, including small proof of concepts. Go is so simple, anyone fluent in PHP, JavaScript or Python can write it without facing a steep learning curve. Rust and Swift both do have a steeper learning curve than Go, but are still nowhere near learning C or C++. The amount of highly performant compiled apps on the Internet is posing a real challenge for applications written in interpreted languages such as PHP, JavaScript, TypeScript and Python.

Interpreters’ performance challenges

To understand the issue, the below application can serve as a demonstration. It performs a very simple task as it fetches the sitemap.xml of a website and then the HTML of each page. It then extracts the OpenGraph information of each page of that website. With less than 100 lines of code, this small function could cause severe damage to any website that is not optimized to scale seamlessly.

/* this is an incomple example of a go function that 
  fetches the OpenGraph information of all web pages
  on a given domain. It uses goroutines and a channel
  with a relatively large buffer of 255. */
func PrintWebsitePages(domainName string) {
 // fetch the sitemap to get all pages on the website
 sitemapURL := "https://" + domainName + "/sitemap.xml"
 sitemap, err := fetchSitemap(sitemapURL)
 if err != nil {
  log.Fatal(err)
 }

 // create a buffered channel to store PageInfo structs
 resultChannel := make(chan PageInfo, 255)

 // use a wait group to wait for all goroutines to finish
 var wg sync.WaitGroup

 // iterate over URLs in the sitemap and start 
 // a goroutine for each to fetch the OG tags
 for _, url := range sitemap.URLs {
  wg.Add(1)
  // just for demonstration purposes, not implemented. This
  // func will fetch the open graph information
  go fetchOpenGraphInfo(url.Loc, resultChannel, &wg)
 }

 // close the channel when all goroutines are done
 go func() {
  wg.Wait()
  close(resultChannel)
 }()

 // read results from the channel and simply
 // print them to the console/terminal
 for pageInfo := range resultChannel {
  fmt.Printf("URL: %s\nTitle: %s\nDescription: %s\n\n", 
      pageInfo.URL, pageInfo.Title, pageInfo.Description)
 }
}

Imagine a PHP, Python or Node.js application on the other side that generates the Sitemap.xml file and each web page on the fly like a traditional 3-tier web app. That website is now faced with an application that hammers it with hundreds of requests in just a few seconds. If that website, like most websites, isn’t hosted on object stores like AWS S3, Google Cloud Storage or Azure Blob Storage, it’ll either have to scale (if on automatically scaling infrastructure) or simply end up in a denial of service.

The problem is not the number of requests, but the cpu and memory resources required to respond to these requests. Even the smallest Linux boxes can easily handle thousands of requests per second. With interpreters often requiring 5 to 10x more cpu and memory resources, this becomes an uneven competition between two system communicating with each other.

It’s not a problem for the compiled

Let’s look at the problem the other way around: imagine a Node.js application performing the same requests to a Go application which serves content with HTMX or Go Gin. Even on small Linux boxes, the Go application won’t have much of a problem quickly responding to a few hundred requests. While a Node application may need to consume gigabytes to respond to these requests, the Go application will only need a tenth of that, probably only a few hundred megabytes.

async function printWebsitePages(sitemapURL) {
  try {
    const result = await fetchSitemap(sitemapURL);
    const urls = result.urlset.url;

    // Use Promise.all to fetch OpenGraph info concurrently for all URLs
    const results = await Promise.all(urls.map(async (urlData) => {
      const url = urlData.loc[0];
      return await fetchOpenGraphInfo(url);
    }));

    // Filter out null results (errors during fetching OpenGraph info)
    const validResults = results.filter((result) => result !== null);

    // Log the results
    validResults.forEach((result) => {
      console.log(`${result.Title}\n${result.Description}\n\n`);
    });
  } catch (error) {
    console.error(`Error: ${error.message}`);
  }
}

While a node application can run tasks in parallel, it’s memory requirements serve as a natural boundary. If you have ever done batch processing with Node, PHP or Python, you’ll know how fast you can run into memory limitations. Far, far sooner than any application written in Rust, Go or Swift. If you want to try it yourself, read “Extracting OpenStreetMap With Go” and try it yourself with PHP or Node.js.

High performance is becoming the norm

Usually, crawlers like the example above would throttle to only a few dozens requests per minute to ensure they won’t harm the remote server when requesting a lot of content from it. Yet, even with a few dozen requests per minute they could severely hurt a badly configured server or a poorly written application. With more and more server side code written in Go, Rust or Swift, higher levels of performance are quickly becoming the norm. Leaving the interpreters of PHP, Node and Python with ever higher memory and cpu requirements when facing the new normal.

Lower performance equals higher operating cost

Even worse for commercial applications. If your commercial service is written in TypeScript, JavaScript, PHP or Python, you’ll need more resources compared to the other languages and thus have 10x higher operating cost. While this doesn’t really matter if your Laravel-based PHP app will only ever serve 50 customers, it’ll become a complete nightmare if that application is set out to serve 5,000 customers.

The resources needed for interpreted apps to scale are completely through the roof when compared to compiled apps. The argument of most PHP developers is that Facebook was written in PHP, which is true. However, Facebook struggled with exactly that years ago. It led Facebook to several attempts to improve performance. These included HHVM and inventing Hacklang. This proves that technically all the interpreted languages can scale nicely, but at a hefty price tag.

Poor performance hurts the environment

Even if your interpreted app runs on renewable energy, it still consumes more energy than a compiled application. Energy that would be needed for other industries to transition to renewables — it’s simply not energy efficient. If your interpreted application has a 10x higher footprint than a compiled one, you’re not doing any good to the environment. There are numerous studies and benchmarks arising on how environmentally friendly which programming language is. Check these studies and you’ll see that interpreted languages are at the bottom of these rankings. They are the unfiltered diesel engine of programming languages.

AOT compilers have a smaller environmental footprint — by the very nature of how they work

It’s simple: a compiled binary (AOT compilation) is only compiled once by the developer or the build pipeline while intepreted code (JIT compilation or interpretation) is compiled or interpreted to machine code whenever the application runs. That means when a user requests a website, the interpreted app is compiled or interpreted to machine code whereas the compiled binary is already in machine code.

Go, Rust and Swift define the new normal

Migrating your app from an interpreted language to a compiled one was usually something you’d do when it grew beyond a certain size. Simply because rewriting in a compiled language was time consuming and thus pricey in terms of labour cost. Most of the time, C/C++ were the only serious option. Swift, Rust and Go have changed that. Many applications today are compiled apps from the start.

Read “Writing server software in Go” or “Serverless Swift With Vapor On AWS Using AWS SAM And Lambda” and you will realize how easy it has become to write compiled apps with multi-threading. The TIOBE index gives you a hint: Go and Rust are having their all-time high this year. Swift on the server is set out to a breakthrough in 2024 with more of the required libraries reaching mature versions. Oracle’s GraalVM, that compiles Java, shows how much the AOT-Compiler made a big time comeback.

It’s an uneven battle

Performance wise, compiled apps interacting with interpreted apps on the Internet are like a dog fight between a fighter jet an a propeller aircraft. On the environmental side, it’s like a solar panel competing with a coal mine.

Eco-friendly rocket vs propeller: Is the demise of interpreters and JIT compilers near?

AOT compilation, that was the norm even before C was invented in 1972, results in applications with significantly higher performance and a much lower environmental footprint. AOT compilers create fast and eco-friendly applications. Something that interpreters and JIT compilers will hardly ever achieve: It’s trying to square a circle.

Personally, I always enjoyed compiled languages more than interpreted ones. When I wrote C or C++ code decades ago it was frustrating. I always wished for a compiled language that does not require me to handle all the bits and pieces of memory management. With the arrival of Go (Garbage collection or “GC”), Rust (Borrow checking) and Swift (Automatic Reference Counting or “ARC”), that wish came true. If you want to know more about thoughts on that, read “How Go fixed everything that was wrong with programming”.

Leaving all emotions and personal preference for interpreted languages such as JavaScript, TypeScript (transliterates to JavaScript), PHP and Python aside: What honest justification do I still have to use interpreted languages?

Thank you for reading. Jan